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To characterize strongly interacting statistical systems within a thermodynamical framework - 
complex systems in particular - it might be necessary to introduce generalized entropies, Sg. A 
series of such entropies have been proposed in the past, mainly to accommodate important em- 
pirical distribution functions to a maximum ignorance principle. Until now the understanding of 
the fundamental origin of these entropies and its deeper relations to complex systems is limited. 
Here we explore this questions from first principles. We start by observing that the 4th Khinchin 
axiom (separability axiom) is violated by strongly interacting systems in general and ask about the 
consequences of violating the 4th axiom while assuming the first three Khinchin axioms (Kf-K3) 
to hold and Sg = XliP(Pi). We prove by simple scaling arguments that under these requirements 
each statistical system is uniquely characterized by a distinct pair of scaling exponents (c, d) in the 
large size limit. The exponents define equivalence classes for all interacting and non interacting 
systems. This allows to derive a unique entropy, Sc,d oc ^- r{d + 1, 1 — clnpi), which covers all 
entropies which respect K1-K3 and can be written as Sg = "^^gipi)- Known entropies can now 
be classified within these equivalence classes. The corresponding distribution functions are special 
forms of Lambert- VK exponentials containing as special cases Boltzmann, stretched exponential and 
Tsallis distributions (power-laws) - all widely abundant in nature. This is, to our knowledge, the 
first ab initio justification for the existence of generalized entropies. Even though here we assume 
Sg — X^i g{Pi)y we show that more general entropic forms can be classified along the same lines. 



Weakly interacting statistical systems can be perfectly 
described by thermodynamics - provided the number of 
states W in the system is large. Complex systems in 
contrast, characterized by long-range and strong interac- 
tions, can fundamentally change their macroscopic qual- 
itative properties as a function of the number of states 
or the degrees of freedom. This leads to the extremely 
rich behavior of complex systems when compared to sim- 
ple ones, such as gases. The need for understanding the 
macroscopic properties of such interacting systems on the 
basis of a few measurable quantities only, is reflected in 
the hope that a thermodynamic approach can also be 
established for interacting systems. In particular it is 
hoped that appropriate entropic forms can be found for 
specific systems at hand, which under the assumption of 
maximum ignorance, could explain sufficiently stationary 
macro states of these systems. In this context a series of 
entropies have been suggested over the past decades, ^ 
la] and Table 1. So far the origin of such entropies has 
not been fully understood within a general framework . 

Here we propose a general classification scheme of both 
interacting (complex) and non- or weakly-interacting sta- 
tistical systems in terms of their asymptotic behavior un- 
der changes of the number of degrees of freedom of the 
system. Inspired by the classical works of Shannon [7[ 
and Khinchin Q we follow a classical scaling approach 
to study systems where the first three Khinchin axioms 
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hold; we study the consequences of the violation of the 
fourth, which is usually referred to as the separation ax- 
iom. The first 3 Khinchin axioms are most reasonable to 
hold also in strongly interacting systems. 

The central concept in understanding macroscopic sys- 
tem behavior on the basis of microscopic properties is 
entropy. Entropy relates the number of states of a sys- 
tem to an extensive quantity, which plays a fundamental 
role in the systems thermodynamical description. Ex- 
tensive means that if two initially isolated, i.e. suf- 
ficiently separated systems, A and B, with Wa and 
Wb the respective numbers of states, are brought to- 
gether, the entropy of the combined system A + B is 
S{Wa+b) = S{Wa) + S{Wb)- Wa+b is the number of 
states in the combined system A + B. This is not to 
be confused with additivity which is the property that 
S{WaWb) = S{Wa) + S{Wb)- Both, extensivity and 
additivity coincide if number of states in the combined 
system is Wa+b = WaWb- Clearly, for a non- interacting 
system Boltzmann-Gibbs entropy, 5'bg[p] = J2i9^GiPi)i 
with gBoix) — —a; In a;, is extensive and additive. By 
'non-interacting' (short-range, ergodic, sufhciently mix- 
ing, Markovian, ...) systems we mean Wa+b = WaWb- 
For interacting statistical systems the latter is in gen- 
eral not true; phase space is only partly visited and 
Wa+b < WaWb- In this case, an additive entropy such 
as Boltzmann-Gibbs can no longer be extensive and vice 
versa. To keep the possibility to treat interacting sta- 
tistical systems with a thermodynamical formalism and 
to ensure extensivity of entropy, a proper entropic form 
must be found for the particular interacting statistical 
systems at hand. We call these entropic forms general- 



ized entropies and assume them to be of the form 

w 



^aip\='^9{Pi) , 



(1) 



W being the number of states^. The four Khinchin ax- 
ioms (K1-K4) uniquely determine g to be the Boltzmann- 
Gibbs-Shannon (BG) entropy Q. These axioms have a 
series of imphcations on g: 

• Kl: The requirement that S depends continuously 
on p implies that 5 is a continuous function. 

• K2: The requirement that the entropy is maximal 
for the equi-distribution pi = 1/W implies that 
g is a concave function (for the exact formulation 
needed in a proof below, see SI Proposition 1). 

• K3: The requirement that adding a zero- 
probability state to a system, W+1 withpvF+i — 0, 
does not change the entropy, implies ^(0) — 0. 

• K4: The entropy of a system - split into sub- 
systems A and B - equals the entropy of A plus 
the expectation value of the entropy of B, condi- 
tional on A. 

If Kl to K4 hold, the entropy is the Boltzmann-Gibbs- 
Shannon entropy, 



w 
Sbg[p\ =^9bg{Pi) 



with gBci^) — — ajlnx. (2) 



i=l 



The separability requirement of K4 corresponds exactly 
to Markovian processes and is obviously violated for most 
interacting systems. For these systems, introducing gen- 
eralized entropic forms Sg [p] , is one possibility to ensure 
extensivity of entropy. We assume in this paper that 
axioms Kl, K2, K3 hold, i.e. we restrict ourselves to 
Sg = J2i9iPi) with g continuous, concave and g{0) = 0. 
These systems we call admissible systems. 

In the following we classify all (large) statistical sys- 
tems where K1-K3 hold in terms of two asymptotic prop- 
erties of their associated generalized entropies. Both 
properties are associated with one scaling function each. 
Each scaling function is characterized by one exponent, 
c for the first and d for the second property. They ex- 
ponents allow to define equivalence relations of entropic 
forms, i.e. two entropic forms are equivalent iff their ex- 
ponents are the same. The pair (c, d) uniquely defines 



Obviously not all generalized entropic forms are of this type. 
Renyi entropy e.g. is of the form G(X]i 9{Pj))i with G a mono- 
tonic function. We use the entropic forms Eq. JTJ for simplicity 
and for their nice characterization in terms of asymptotic prop- 
erties. Using the Renyi form, many asymptotic properties can be 
studied in exactly the same way as will be shown here, however 
it gets technically more involved since asymptotic properties of 
G and g have to be dealt with simultaneously. 
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FIG. 1: Equivalence classes of functions gc,d parametrized in 
the (c, d)-plane, with their associated entropies and charac- 
teristic distribution functions. Following [g, [13] entropies are 
one-to-one related to distribution functions. BG entropy cor- 
responds to (1, 1), Tsallis entropy to (c, 0), and entropies for 
stretched exponentials to (l,d > 0). All entropies leading to 
distribution functions with compact support, belong to equiv- 
alence class (1,0). An example are Sq entropies with g > 1 
(using the maximum entropy principle with usual expectation 
values in the constraints 



ropy pri 



an equivalence class of entropies. Each admissible sys- 
tem approaches one of these equivalence classes in its 
W ^>- 00 limit. 

To be very clear, by asymptotic we mean the number of 
states being large, IF 3> 1. Thus all the relevant entropic 
information on the system is encoded in the properties 
of g{x) near zero, i.e. in the region x ~ W~^. In the 
asymptotic limit it is therefore not necessary to know g on 
the entire interval of a; € [0, 1], but it is sufficient to know 
it in the vicinity of a; ~ 0. In other words the part of g{x) 
where x > W~^ contains information which is irrelevant 
for the macroscopic properties. In terms of distribution 
functions this simply means that everything but the tails 
becomes irrelevant for large systems. This implies that 
the equivalence classes (c, d) can be interpreted as basins 
of attraction for systems that may differ on small scales 
but start to behave identical in the thermodynamic limit. 

We show that a single two-parameter family of en- 
tropies Sc,d oc X)i r(d + 1, 1 — clnpi), is sufficient to 
cover all admissible systems; i.e. all entropies of the 
form Sg = J^idiPi) ^^^ equivalent to some represen- 
tative entropy Sc.d, which parametrizes the equivalence 
classes (c, d). Distribution functions associated with Sc.d 
involve Lambert-IF exponentials. Lambert-VF functions 
have deep connections to self-similarity and time-delayed 
differential equations, see e.g. [ii|,[l3- Important special 
cases of these distributions are power-laws (Tsallis en- 
tropy [9|) and stretched exponential distributions which 
are widely abundant in nature. 



I. ASYMPTOTIC PROPERTIES OF 
NON-ADDITIVE ENTROPIES 

We now discuss 2 scaling properties of generalized en- 
tropies of the form S = '^^gipt) assuming the validity of 
the first 3 Khinchin axioms. 

The first asymptotic property is found from the scaling 
relation 



Sg(XW) _,9{jk^) 



a: 



(3) 



in the limit W -^ oo, i.e. by defining the scaling function 

(4) 



f{z) = lim„ ^^ (0 < z < 1) 



>0 g{x) 

The scaling function / for systems satisfying K1,K2, K3, 
but not K4, can only be a power f(z) — z"^, with < 
c < 1, given / being continuous. This is shown in the SI 
(Theorem 1). Inserting Eq. ((4]) in Eq. (jSj gives the first 
asymptotic law 



hm ¥^ = A-^ 



W-^OO Sg{W) 

From this it is clear that 



lim 



^(^^)a-i = i 



W^oo S{W) 



(5) 



(6) 



If we substitute A in Eq. (|6l) by A ^ W^ we can identify 
a second asymptotic property. We define hc{a) 

^ ' w^oo S{W) x^o x<^'=g{x) ^ ' 

with X = 1/W. hc{a) in principle depends on c and a. It 
can be proved (SI, Theorem 2) that hc{a) is given by 



hc{a) ~ (1 + a)'' {d constant) 



(8) 



Remarkably, he does not explicitly depend on c anymore 
and hc{a) is an asymptotic property which is independent 
of the one given in Eq. (IS|). Note that if c = 1, concavity 
of g implies d > 0. 



II. CLASSIFICATION OF STATISTICAL 

SYSTEMS 

We are now in the remarkable position to characterize 
all large K1-K3 systems by a pair of two exponents (c, d), 
i.e. their scaling functions / and he. See Fig. [TJ 

For example, for gBci^) — — a;ln(x) we have f{z) = 
z, i.e. c = 1, and hc(a) = 1 + a, i.e. d ~ 1. Sbg 
therefore belongs to the universality class (c, d) — (1,1). 
For gq{x) — {x — x'>)/{l — q) (Tsalhs entropy) and < 
q < 1 one finds f{z) — z'^ , i.e. c — q and hc{a) = 1, i.e. 



d = 0, and Tsallis entropy, Sq, belongs to the universality 
class (c, d) — {q, 0). A series of other examples are listed 
in Table 1. 

The universality classes (c, d) are equivalence classes 
with the equivalence relation given by: ga = g/} -i^ Ca = 
cp and da = djj. This equivalence relation partitions the 
space of all admissible g into equivalence classes com- 
pletely specified by the pair (c, d). 



III. THE DERIVATION OF ENTROPY 

Since we are dealing with equivalence classes (c, d) we 
can now look for a two-parameter family of entropies, 
i.e. functions gc,di such that g^d is a representative of 
the class (c, d) for each pair c G (0, 1] and d G M. A 
particularly simple choice which covers all pairs (c, d) is 



9c,dA^) == "^^ e^r(l + d, yl - clnx) - rex 



(9) 



with A 



cdr 



i_(i„e)r - r(a, &) = J^ dtt"^ ^exp{-t) is the 
incomplete Gamma-function and r is an arbitrary con- 
stant r > (see below). For all choices of r the function 
gc,d.r is a representative of the class (c, d). This allows to 
choose r as a suitable function of c and d. For example 
choose r — {I ~ c + cd)^^ , so that A ^ I, and 



Sc,d[p] 



eE..r(l + d, l-clnpQ 
1 — c + cd 



1 — c + cd 



(10) 

The proof of the correct asymptotic properties is found 
in SI (Theorem 4). 



IV. SPECIAL CASES OF ENTROPIC 
EQUIVALENCE CLASSES 

Let us look at some specific equivalence classes. 

• Boltzmann-Gibbs entropy belongs to the (c, d) = 
(1, 1) class. One immediately verifies from Eq. © 
that 



SiAp] = ^9i,iiPt) = -^Pi^^^Pi + 1 



(11) 



• Tsallis entropy belongs to the (c, d) = (c, 0) class. 
With Eqs. ^ and dTT]) we get 



Sc,q[p\ = J2^9c,o{P^) = 



_ 1-E. 



c-1 



1 



(12) 



Note, that although the pointwise limit c — > 1 
of Tsallis entropy is the BG-entropy, the asymp- 
totic properties (c, 0) do not change continuously 
to (1, 1) in this limit! In other words, the thermo- 
dynamic limit and the limit c — > 1 do not commute. 

• An entropy for stretched exponentials has been 
given in f2| which belongs to the (c, d) = {l,d) 



TABLE I: Comparison of several entropies for which S — X^j <?(pi)i ^^'^ K1-K3 hold. They are shown as special cases of the 
entropy given in Eq. (|9]). Their asymptotic behavior is uniquely determined by c and d. It can be seen immediately that 5'g>i, 
Sb and Se are asymptotically identical. So are Sg<i and 5'^ as well as S,, and S-,. 
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5/3(p) = E.pfln(l/pO 


c = p 


1 


[10] 



classes, see Table 1. It is impossible to compute the 
general case without explicitly using the Gamma- 
function. As one specific example we compute the 
(c, d) = (1,2) case, 



51,2 [p] =2(1 -EjP^lnpO + 5Ei-Pi(lnp» 



(13) 



The asymptotic behavior is dominated by the sec- 
ond term. 

• All entropies associated with distributions with 
compact support belong to (c, d) = (IjO). Clearly, 
distribution functions with compact support all 
have the same trivial asymptotic behavior. 

A number of other entropies which are special cases of 
our scheme are listed in Table 1. 



V. THE DISTRIBUTION FUNCTIONS 

Distribution functions associated with the F-entropy, 
Eq. (flOl) . can be derived from the so-called generalized 
exponentials, p(e) = £^c,d,r(— e)- Following [6|,|13| (see also 
SI generalized logs), the generalized logarithm A can be 
found in closed form 



Ac,d,r(a;) = rx^ 



l-(l-c)r ^ 

1 m X 

rd 



(14) 



and its inverse function, £ ^ K ^ ,is 



,^_^_^(^)^^-x^[m'.(b(i-.A)^)-w.(b)]^ ( 



15) 



with the constant B = 



_ (l-c)r 



l-(l-c)r 



exp 



\^l-(l-c)r 



The 



function Wk is the fc'th branch of the Lambert-W^ 
function, which is a solution of the equation x = 
W{x) exp(W^(a;)). Only branch fc = and branch fc = — 1 
have real solutions Wk ■ Branch A: = is necessary for all 
classes with d > 0, branch k = —I for d < 0. 



A. Special cases of distribution functions 

It is easy to verify that the class {c,d) = (1, 1) leads 
to Boltzmann distributions, and the class (c, d) = (c, 0) 
yields power-laws, or more precisely, Tsallis distributions 
i.e. q-exponentials. 

All classes associated with {c,d) = (1, d), for d > are 
associated with stretched exponential distributions. To 
see it, remember that d > requires the branch fc = of 
the Lambert- VF function. Using the expansion Wo{x) ~ 
X — x'^ + . . . for 1 ^ |a;|, the limit c -^ 1 turns out to be 
a stretched exponential 



lim £c,dA^) 

c->-l 



dr (l-f )3_i 



(16) 



Clearly, r does not effect its asymptotic properties, but 
can be used to modify finite size properties of the distri- 
bution function on the left side. Examples of distribution 
functions are shown in Fig. [51 



B. A note on the parameter r 

In Eq. (Uni) we chose r = {1 - c + cdy^. This is 
not the most general case. More generally, only the fol- 
lowing limitations on r are required if the corresponding 
generalized logarithms (for definition see SI) are wanted 
to be endowed with the usual properties (A(l) = and 
A'(l) = l), 



d>0 
d = 
d<0 



r < 



r > 



l-c 

1 

l-c 

1 



(17) 



Note that every choice of r gives a representative of the 
equivalence class (c, d), i.e. r has no effect on the asymp- 
totic (thermodynamic) limit, but it encodes finite-size 
characteristics. A particular practical choice for r is 
r = (1 — c + cd)^^ for d > and r = exp(— d)/(l — c) for 
d< 0. 
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FIG. 2: Distribution functions based on the 'Lambert expo- 
nential', p{x) = £c,d,r{—x) are plotted for various (c, d) values, 
(a) Asymptotically stable systems, i.e. the stretched exponen- 
tial limit c ^ 1. It includes the Boltzmann distribution for 
d = 1. (b) The d — ;■ limit - i.e. the g-exponential limit. In 
(a) and (b) the black dashed lines represent the stretched ex- 
ponential (c = 1) or (jT-exponential {d = 0) limit functions, (c) 
The general case for distribution functions for various values 
of (c, d) away from the limits c ~ 1 or d ~ 0. They should 
not be confused with power-laws. 



VI. A NOTE ON RENYI ENTROPY 

Renyi entropy is obtained by relaxing K4 to a pure 
additivity condition, and by relaxing S = "^ g. For 
Renyi-type entropies, i.e. S — G{J2i=i9{Pi))j one gets 
\imw^ooS{XW)/S{W) = lim,^o,G(A/g(A-i)s)/G(s), 



where fg{z) — Mmx^o 9{zx)/g{x). The expression 
fds) = liuis G{sy)/G{y), now provides the starting 
point of a deeper analysis, which follows the same lines 
as those presented here. However, this analysis gets more 
involved and properties of the entropies get more compli- 
cated. In particular, Renyi entropy, G{x) = ln(2;)/(l — a) 
and g{x) = x", is additive, i.e. asymptotic properties, 
analogous to the ones presented in this paper, would 
yield the class {c,d) = (1,1), which is the same as for 
BG-entropy. However, Renyi entropy can also be shown 
not to be Lesche stable |14| - [l8j . This must not be con- 
fused with the situation presented above where entropies 
were of the form S = J29- ^^ of the S — J2 9 entropies 
can be shown to be Lesche stable (see proof SI Theorem 
3). 



VII. DISCUSSION 

We argued that the physical properties of macroscopic 
statistical systems being described by generalized en- 
tropic forms (of Eq. ([l}) can be uniquely classified in 
terms of their asymptotic properties in the limit W —)' oo. 
These properties are characterized by two exponents 
(c, d), in nice analogy to critical exponents. These ex- 
ponents define equivalence relations on the considered 
classes of entropic forms. We showed that a single en- 
tropy - parametrized by the two exponents - covers all 
admissible systems (Khinchin axioms 1-3 hold, 4 is vio- 
lated). In other words every statistical system has its pair 
of unique exponents in the large size limit, its entropy is 
given by ^e,d ~ E, T (1 + d , 1 - clnp,) Eq. ^. 

As special cases Boltzmann-Gibbs systems have 
{c,d) — (1, 1), systems characterized by stretched expo- 
nentials belong to the class {c,d) = {l,d), and Tsallis 
systems to {c,d) = ((7,0). The distribution functions of 
all systems (c, d) are shown to belong to a class of ex- 
ponentials involving Lambert-iy functions, given in Eq. 
psp . There are no other options for tails in distribution 
functions other than these. 

The equivalence classes characterized by the exponents 
c and d, form basins of asymptotic equivalence. In gen- 
eral these basins and their representatives will character- 
ize interacting statistical (non-additive) systems. There 
is a remarkable analogy between these basins of asymp- 
totic equivalence and the basin of attraction of weakly 
interacting, uncorrelated systems subject to the law of 
large numbers, i.e. the central limit theorem. Although, 
strictly speaking, there is no limit theorem which selects a 
specific representative within any such equivalence class, 
it is clear that any system within a given equivalence class 
may exhibit individual peculiarities as long as it is small. 
Yet systems of the same class will start behaving identi- 
cally as they become larger. Finally, only the asymptotic 
properties are relevant. Distribution functions converge 
to those functions uniquely determined by (c, d). 

Our framework clearly shows that for non-interacting 
systems c has to be 1. Setting A = Wb in Eq. ^ 



and Eq. g]), immediately implies S{WaWb)/S{Wa) ~ 
Wg'^'^. This means that if for such a system it would 
be true that c 7^ 1, then adding only a few independent 
states to a system would explosively change its entropy 
and extensivity would be strongly violated. A further 
interesting feature of admissible systems is that they all 
are what has been called Lesche stable, systems (proof 
in SI Theorem 3). As a practical note Lesche stability 
corresponds one-to-one to the continuity of the scaling 



function / (see SI) and can therefore be checked by a 
trivial verification of this property (Eq. @). 

We have developed a comprehensive classification 
scheme for the generic class of generalized entropic forms 
of type S = X]i5(P*)i ^^^ commented on how the phi- 
losophy extends to entropies of e.g. Renyi type, i.e. 
•S" = C!{J2i g{Pi))- Finally, we argue that complex statis- 
tical systems can be associated with admissible systems 
of equivalence classes (c, d) , with < c < 1 . 
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Supplementary Information 

This supplement to the paper 'A classification of com- 
plex statistical systems in terms of their stability and 
a thermodynamical derivation of their entropy and dis- 
tribution functions' contains detailed information on the 
technical aspects of the work. In particular it contains 
the proofs omitted from the paper for readability. 



Proposition 1 

The consequence of K2 - that the maximal uncon- 
strained entropy is found for equi-distribution pi = 1/W 
- is equivalent to the requirement that 17 is a concave 
function on [0, 1]. This is summarized in the well known 
proposition 

Proposition: 
Let Sg be given by Eq. (1) (main text) and let <; be a con- 
cave function which is continuously diffcrentiable on the 
semi-open interval (0, 1] then ^^(T^^) = maxj^ p.^i 5'g[p], 

is given by Sg{W) = Wg{l/W). 

Proof. Let W be the number of states i = 1, . . . , W. The 
constraint that p is a probability X]i=i Pi ~ 1 '-^^^ be 
added to Sg with by using a Lagrangian multiplier. I.e. 
differentiation of Sg\p] — Q:(^jPi — 1) with respect to pi 
gives g'{pi) = a, where a is the Lagrangian multiplier. 
Since g is concave g' is monotonically decreasing and 
therefore pi = pj for all i and j. Consequently pi — 1/W 



^w 



for all i and J27=i 9{^/W) = Wg{l/W). 



D 



Theorem 1 and proof 

Theorem 1: Let g be a continuous, concave function 
on [0, 1] with g{0) — and let f{z) = lim2.^o+ 9{zx)/g{x) 
be continuous, then / is of the form f{z) = z'^ with c G 
(0,1]. 

Proof. Note that f{ab) — \mix^og{abx)/g{x) ~ 
\im.,^o{g{abx)/g{bx)){g{bx)/g{x)) = f{a)f{b). All 
pathological solutions are excluded by the requirement 
that / is continuous. So f{ab) = f{a)f{b) implies that 
f{z) = z'^ is the only possible solution of this equation. 



Further, since g{0) = 0, also liuix^o g{Ox) / g{x) = 0, and 
it follows that /(O) = 0. This necessarily implies that 
c > 0. f{z) — z^ also has to be concave since g{zx)/g{x) 
is concave in z for arbitrarily small, fixed a; > 0. There- 
fore c < 1. n 

Note that if / is not required to be continuous, then 
there are various ways to construct (rather pathological) 
functions / solving f{ah) = f{a)f{b) different from z'^, 
as for instance f(z) — 1 for z being a rational number 
and /(z) = for z being an irrational number, which is 
nowhere continuous. Also f{z) = limc_>o- ^'^i which is 
zero for z = and one otherwise, would be a possible 
solution. The continuity requirement eliminates all these 
possibilities. 



Theorem 2 and proof 

Theorem 2: Let g be like in Theorem 1 and let f{z) = 
z'^ then he given in Eq. (8) is a constant of the form 
hcia) = (1 + af^ for some constant d. 

Proof. We can determine hc{a) again by a similar trick 
as we have used for /. 



hc{a) ^lim^^of^l^ 



.((.-)(^-0+-) 



gj^') 



{x") 



i^-^>,(x 



g{x>') 



"gi^x) 



for some constant b. By a simple transformation of vari- 
ables, a — 66' — 1, one gets hc{bb' — 1) = hc{b — l)hc{b' — l). 
Setting H{x) — hc{x — 1) one again gets H{bb') = 
H{b)H{b'). So H{x) = x'^ for some constant d and con- 
sequently hc{a) is of the form (1 + a)'*. D 



Theorem 3 on Lesche stability and the theorem 
relating it to continuous /, and its proof 

The Lesche stability criterion is a uniform-equi- 
continuity property of functionals S[p\ on families of 
probability functions {p'^^^]w=i where p''^) = {pf]"^!- 
The criterion is phrased as follows: 

Let p'^^^ and q'^'^^ be probabilities on W states. An 
entropic form S is Lesche stable if for all e > and all 
W there is a J > such that 

where S{W) is again the maximal possible entropy for 
W states. We now characterize Lesche stability on the 
class of the generalized entropic forms in terms of the 
continuity of / in 



Theorem 3: Let pi > be a probability, i.e. 
J2i=iPi — 1' ^^d W the number of states i. Let g be 
a concave continuous function on [0, 1] which is continu- 
ously differentiable on the semi-open interval (0, 1]. Also, 
let g{0) = then the entropic form Sg[p] = J2i=i diPi) is 
Lesche stable iff the function f(z) = lim.x-^o g{zx) / g{x) 
is continuous on z G [0, 1]. 

Proof. Proposition 1 states that the maximal entropy is 
given by Sg{W) = Wg{l/W). We now identify the worst 
case scenario for | Sg [p] — Sg [q] \ , where p and q are prob- 
abilities on the W states. This can be done by maximiz- 
ing G[p,g] = \Sg[p] - Sg[q]\ - a(E.P» - 1) - /^(E.ft " 
1) ~ 7(Si \Pi ~ 1i\ ~ ^)j where a, (3 and 7 are Lagrange 
multipliers. Without loss of generality assume that 
Sg[p] > Sg[q] and therefore the condition dG/dpi — 
gives g' (pi) + J sign{pi —qi)—a — 0, where g' denotes the 
first derivative of g; sign is the signum function. Simi- 
larly, dG/dqi = leads to g'{qi) + 7 sign{pi — qi)+ (3 — 0. 
From this we see that both p and q can only possess two 
values p-(-, p- and (7+ and g_ , where we can assume (with- 
out loss of generality) that p+ > g_ and q+ > P- • We 
can now assume that for w indices i p+ = pi > qi = g_ 
and for W — w indices j p- = pj < qj = q+ where w may 
range from 1 to VK — 1. This leads to seven equations 

wp^ + {W — w)p^ — 1 , g'{p+) + 7 — a = 
wq^ + [W — w)q+ — 1 , g'{p-) — 7 — a = 

w{p+-q-)- , 9'{p+) + l + (3 ^0 

-{W - w){p^ -q+)=S , g'{p+) - 7 + /3 = 

which allow to express p_ , g_ , and q^ in terms of p-|_ 

p~ — {I ~ wp+) / {W — w) 

g_ = p+ — S/2w 

q+= {l-wp+)/{W-w)+6/2{W-w) . 



Further we get the equation 

g'{p+)-g'{p-) + g'{q+)-g'{q-)^0 



(18) 



However, since g is concave g' is monotonically decreasing 
and therefore 5' (p+) — (7'((j'_) > a.nd g'{q+)—g'{p-) > 0. 
Thus Eq. p8)) has no solution, meaning that there is no 
extremum with p± and q± in (0, 1), and extrema are at 
the boundaries. The possibilities are p+ = 1 or p_ = 0, 
then (7_|_ = 1 and q^ = 0. Only P4- = 1 or p_ = are 
compatible with the assumption that S[p] > S[q] (the 
other possibilities are associated with S[q] > S[p]); p+ = 
1 is only a special case of p_ =0 with n = 1. Since 
(7(0) =0 this immediately leads to the inequality 



Smax 



<(1 



.K, 



g{l/W) 



9\ 4>w ) _ 9\ ipw ) 



gil/W) g{l/W) 



where </> = w/W is chosen such that the right hand side of 
the equation is maximal. Obviously for any finite W the 
right hand side can always be made as small as needed 



by choosing S > small enough. Now take the limit 
W —^ oo. If / is continuous and using Theorem 1 



\Sg[p]-S,lq]\ ^ 

<(1-'^)(2(T^) 

< {1 - ^y-'s^ + (t 



a)'-(^ 



-5/2 



<<5" + |l 



(l-c<5/2)| 



(19) 



It follows that Sg is Lesche-stable, since we can make 
the right hand side of Eq. ([T51) smaller than any given 
e > by choosing S > small enough. This completes 
the first direction of the proof. If, on the other hand, Sg 
is not Lesche-stable then there exists an e > 0, such that 



\Sg\p] 



Sg[q]\/Sn 



> e, ViV. This implies 



a-^)f{ 



2{r~4 



/(i)-/( 



1-5/2 



> e 



V(5 > 0. This again means that either f{z) is discontin- 
uous at z = l/(j> or limz^o f{z) > 0. Since 17(0) = 
implies that /(O) = 0, f{z) has to be discontinuous at 
z = 0. D 



• The usual logarithm A — log has these properties. 

• The dual logarithm A* (a;) = — A(l/a;) also obeys 
the the conditions. So, if A(a;) can be constructed 
for X G [0, 1], then A can be continued to a: > 1 by 
defining A(x) = A*(a;) for a; > 1 and A is automat- 
ically continuous and differentiable at a; = 1. 

• If systems A and B have entropic forms 6*3^ and 
Sg^ which are considered in a maximum entropy 
principle then the resulting distribution functions 
PAi = £A{-ciA-fiAe-i) andpBi = SBi-aB-PBf-i), 
where £a/b — ^^/b ^'^^ ^^^ generalized exponen- 
tial functions, then the values of a and /3 of system 
A and B are directly comparable. 

Note that to fulfill Eq. (pn|) it may become necessary 
to introduce a constant r, as we have done in the main 
text. 



Proof of asymptotic properties of the 
Gamma-entropy 



Note that if g{x) is differentiable at a; = 0, then 
as a simple Lemma of Theorem 3 it follows that Sg 
is Lesche stable, since f{z) = WuYx^o g{zx) / g{x) = 
{g'{0)zx)/{g'{Q)x) -^ z. Consequently, all g ana- 
lytic on [0,1] are Lesche stable. Moreover, hi{a) — 
ig'{0)x''+^)/{x''g'{0)x) = 1 = (1 -^ 0)°. All these g fah 
into the equivalence class {c,d) = (1,0). 

As an example for a practical use of the above lemma, 
let us consider a function g{x) oc ln(l/x)~^ for a: '^ 0. 
Clearly for z > we have f{z) — \imx^o9izx)/g{x) = 
ln(a;)/ln(za;) — )• 1. On the other hand for z = we find 
f{z) = 0. Therefore, f{z) is not continuous at z = 
and is violating the preconditions of Theorem 3. Lesche 
instability follows as a Lemma and does no longer require 
a lengthly proof. 



Defining generalized logarithms 

It is easy to verify that two functions gA and gB give 
rise to equivalent entropic forms, i.e. their asymptotic 
exponents are identical, if lim^^o+ 9A{x)/gB{x) — (f> and 
is a positive finite constant 00 > > 0. There- 
fore transformations of entropic forms 5*^ of the type 
g{x) — >■ ag{bx), with a > and b > positive constants, 
lead to equivalent entropic forms. Following [y, [l3| . 
the generalized logarithm Ag associated with the en- 
tropic function g is basically defined by —g'{x). How- 
ever, to guarantee that scale transformations of the type 
g{x) —^ o,g{bx) do not change the associated generalized 
logarithm, Ag, one has to define Ag{x) = —ag'{bx), where 
constants a and b are fixed by two conditions 



(i) A(l) = and (h) A'(l) == 1 . (20) 

There are several reasons to impose these conditions. 



The entropy based on gc.d,r, Eq. (10) (main text), 
indeed has the desired asymptotic properties. 

Theorem 4: Let g be like in Theorem 3, i.e. let 
/(z) = z'' with < c < 1, then 



hm = c . 



(21) 



Proof. Consider 



lim 



g(x)-g(^m) 
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x^0+ 



Taking the limit z — !■ 1 on both sides completes the proof. 

D 

Further, two functions gA and gB generate equivalent 
entropic forms if \ub.^^q+ gA{x)/ gB{x) = (f> and < (/> < 
cxD. This clearly is true since 



lim , 9a{zx) 



gAJzx) gBJx) gBJzx) 
gB(zx) gA(x) gnix) 
^^-igsizx) 
^^ gB[x) 

lim , SBJzx) 



93 {x) 

By an analogous argument the same result can be ob- 
tained for the second asymptotic property, Eq. (8) (main 
text). A simple lemma is that given gB{x) = agA{bx), 
for some suitable constants a and 6, then gB and gA are 
equivalent. 

A second lemma, following from Eq. (f2T|) is that 



^^-^0+ gB{x) x^o+ g'g{x) 



lim 



which is just the rule of L'Hospital shown to hold for 
the considered families of functions g. This is true since, 
either lim2._^o+ 9a{x)/ gB{x) — (j) with < (f> < oo and 
CA = cb, i.e. gA and gs are equivalent, or gA and gs are 
inequivalent, i.e. ca 7^ cb but (j) = or 4> ^- 00 . 

So if one can find a function (/tost, having the desired 
asymptotic exponents c and d, it suffices to show that 
< -lim^^o+ Ac,d,r (a;) /fftest (a;) < 00, where Kc,d,r is 
the generalized logarithm Eq. (15) associated with the 
generalized entropy Eq. (10) (main text). The test func- 
tion 5tost(a;) — x^log(l/x)'^ is of class (c, d), as can be 
verified easily. Unfortunately gtest can not be used to 
define the generalized entropy due to several technicali- 
ties. In particular (/test lacks concavity around x ^ 1 for 
a considerable range of (c, d) values, which then makes 



it impossible to define proper generalized logarithms and 
generalized exponential functions on the entire interval 
X € [0, 1]. However, we only need the asymptotic proper- 
ties of (/tost and for a; ^ the function (/tost does not vio- 
late concavity or any other required condition. The first 
derivative is g[^^^^{x) = a;^^-'^ log(l/a;)''^-'^(clog(l/a;) — d). 
With this we finally get 



A 



c.d.r 



ix) 



c-1 



r-D-^^)- (log(f)) 



lini 

-^0+ gi,,^{x) x^-i log(l/x)'^-i(clog(l/x) - d) 



vl-c 



cC 



Since < ^jyj- < 00 this proves that the Gamma-entropy 
9c.d,r, Eq- (10) (main text), represents the equivalence 
classes (c, d). 



